REMARKS 

Reconsideration of the above-identified patent application in view of the 
amendments above and the remarks following is respectfully requested. 

Claims 1-12 are in this case. Claims 1-12 have been rejected under § 102(e). 
Independent claims 1 and 11 have been amended. New claims 13 and 14 have been 
added. 

The claims before the Examiner are directed toward a system and method to 
provide feedback to an operator of a device having a delay in the feedback path. The 
operator is provided with a display device operative to display an image from a 
camera located on the device and, via a control device such as a joystick, the operator 
issues commands to the device that can include commands to move the device or to 
move the camera relative to the device or to zoom the lens of the camera. To prevent 
problems associated with feedback delay, such as a tendency to oversteer, on the one 
hand, or to be overly cautious and slow, on the other hand, the present invention 
displays a predicted image on the operator's display that closely approximates the 
view that would be expected from the camera if there were no feedback delay. 

§ 102(e) Rejections 

The Examiner has rejected claims 1-12 under § 102(e) as being anticipated by 
lida et al., US Publication No. 2002/0180878 (henceforth, "lida et al '878")* The 
Examiner's rejection is respectfully traversed. 

lida et al. '878 teach a camera control system that presents predictive feedback 
to an operator via a display. The objective of this predictive feedback, as in the 
present invention, is to allow the operator to judge the effects of movement 
commands that the operator has issued before actual image feedback is available from 
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the camera. The predictive feedback is presented to the operator via markings on 
scroll bars or meters or similar visual devices, or via frames drawn on the display. 

Following is a citation from paragraphs [0077-0079] of lida et aL £ 878, 
describing the display of predictive feedback in the First Embodiment: 

[0077] FIG. 9 and FIG. 10 are views showing examples of 
screen displayed on the image display section 32 during the display 
process of the portable phone 3 according to the present invention. An 
image 40 shown in FIG, 9 is an image which is captured in the 
condition that an optical operation command is inputted and the 
driving section 23 of the camera 2 is driving. Scroll bars 400, 410, 420 
in FIG. 9 represent respective drivable ranges of panning, tilting and 
zooming of the driving section 23 of the camera 2, The drivable range 
of the driving section 23 in each scroll bar is generated based on the 
drivable range of the driving section 23 preliminarily stored in the 
memory 37 of the portable phone 3. 

[0078] Further, black markers 401, 411, 421 on the respective 
scroll bars represent a pan position, a tilt position and a zoom position 
when the image 40 is captured, and are calculated based on view 
position data 102 added to the image data 110 from which the image 
40 was generated. Further, shaded markers 402, 412, 422 respectively 
show target positions of pan position, tilt position and zoom position of 
the driving section 23 calculated from the target position data 111. 

[0079] When the portable phone 3 receives new image data 
110, view position data 102 to be added to the image data 110 is 
obtained in step S27 (FIG. 7), and the markers 401, 411, 421 are 
displayed on the image display section 32 in displaying the new image. 
When the user inputs an optical operation command, target position 
data 111 is newly rewritten in step S25, and the new target position is 
displayed by means of the markers 402, 412, 422. (sic, emphasis 
added) 

Following is a further citation from paragraphs [0088-0090] of lida et aL 6 878, 
describing the display of predictive feedback in the Second Embodiment: 

[0088] FIG. 11 and FIG. 12 are views showing examples of 
screen displayed on the image display section 32 in a display process 
of the portable phone 3 according to the present embodiment. An 
image 50 shown in FIG. 1 1 is an image captured in the condition that 
an optical operation command is inputted from the operational section 
31 and the driving section 23 of the camera 2 is driving. Furthermore, 
meters 500, 510, 520 represent drivable ranges of panning, tilting and 
zooming of the driving section 23 of the camera 2, respectively. 

[0089] Furthermore, needles 501, 511, 521 denoted by the 
black bold lines on the meters respectively show a pan position, a tilt 
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position and a zoom position at the time of capture of the image 50, 
and are calculated on the basis of the view position data 102 added to 
the image data 110 from which the image 50 is generated. Further, 
needles 502, 512, 522 denoted by the dotted lines respectively show 
target positions of the pan position, the tilt position and the zoom 
position of the driving section 23 on the basis of the target position 
data 11. 

[0090] An image 51 shown in FIG. 12 is an image which is 
captured in the condition that the driving section 23 has completed 
driving. Further, each meter in FIG. 12 is as same in meaning as that 
denoted by the same reference numeral in FIG, 11, and needles 503, 
513, 523 denoted by the black bold lines on the meters respectively 
show a pan position, a tilt position and a zoom position at the time of 
capture of the image 51 based on the view position data 102 added to 
the image data 110 from which the image 51 is generated, (sic, 
emphasis added) 

The following citation from page 7 line 9 through page 8 line2 of the present 

application makes clear the distinction between the present invention and the First and 

Second Embodiments of lida et al. c 878: 

It should be noted that image 30 is created by altering image 28 
to reflect the predicted view from camera 18 when device 14 is at the 
second position and not by superimposing an arrow or vector on image 
28 to show where the second position is. (emphasis added) 

Applicant respectfully contends that it would be clear to one skilled in the art 
that visual devices such as the markings on scroll bars of the First Embodiment of lida 
et al. 4 878 or the meters of the Second Embodiment of lida '878 require more 
interpretation on the part of the operator than a display showing an estimate of the 
current view from the camera based on a previous view from the camera and modified 
so as to account for changes in that view that would be caused by movement 
commands issued up to that time, as taught in the present application. 

Following is a citation from paragraphs [0098-0100] of lida et aL '878, 

describing the display of predictive feedback in the Third Embodiment: 

[0098] Further, based on the target position data, the portable 
phone 3 displays the target position as a frame representing a capture 
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range after driving of the camera 2 (step S308) to complete the display 
process. 

[0099] FIGS. 14 and 15 are views showing examples of screen 
displayed on the image display section 32 in the display process of the 
portable phone 3 in accordance with the present embodiment. A range 
6 shown in FIG. 14 represents the full range of the camera 2, and an 
image 60 is the latest image displayed based on the image data 110 for 
which reception of one frame data has completed. Further, a view 
frame 600 is a frame representing a view position generated on the 
basis of the view position data acquired from the image data 110 
currently being received (namely, corresponding to a capture range of 
the image to be displayed next), and a predictive frame 601 is a frame 
representing a target position after driving generated on the basis of 
the target position data (namely, corresponding to a capture range 
after driving), 

[0100] As a result of the above, the image processing section 
340 can set the display area of the image display section 32 based on 
the full range of the camera 2 and display in the display area capture 
ranges of the camera 2 at the target position and the view position, 
making it possible to improve the operability of the user as is the case 
of the first embodiment, (sic, emphasis added) 

Although the above-cited Third Embodiment of Iida et al. '878 does provide a 
more intuitive form of predictive feedback than the First and Second Embodiments of 
Iida et al. '878, wherein a frame drawn on the display is operative to show the 
operator a prediction of the camera view after the motion control commands issued up 
until the present time have been completed, the image only occupies a fraction of the 
available display area because the display is formatted to include the full range that 
can be covered by panning and tilting the camera, As an example, if the camera has a 
horizontal viewing angle of 90° and the camera can be panned so as to cover 270° 
horizontally, the camera image will only occupy one third of the width of the display 
and provide the operator with correspondingly poorer resolution than would be 
provided by using the full width of the display. 

Zooming exacerbates the above-described problem. If the display is to 
correspond to the total area that can be viewed by the camera by panning, tilting and 
zooming, then zooming the lens to a telephoto focal length would result in the image 
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provided by the camera occupying a small frame on the operator's display, with the 
rest of the display area substantially unused. 

Furthermore, such a display is problematic if the camera is mounted on a 
platform that permits unrestricted rotation, as would be the case where the camera is 
mounted on a ground vehicle or an aircraft, because there are no absolute limits on the 
rotation of the camera to define the boundaries of the display. 

In a system where the camera is mounted on a moving vehicle it is highly 
desirable to present the operator with a view that substantially provides the impression 
of being in the vehicle and looking along the optical axis of the camera. The moving- 
frame display taught by lida et aL c 878 does not provide this perspective. 

A display according to lida et aL 4 878 can be useful as a viewfmding device 
for guiding a camera in a situation where the operator display is not intended as the 
primary display for the image captured by the camera, as in the case of a film camera 
or a television camera with a separate display upon which the image from the camera 
occupies substantially the full area of the separate display. However, Applicant 
respectfully contends that the above-described drawbacks of the operator display 
described by lida et aL '878 render this system significantly less satisfactory than that 
described by the present invention for applications such as the remote control of 
ground vehicles and aircraft. 

By contrast, the device of the present invention provides the operator with the 
most up-to-date estimate of the view from the remote camera possible given the time- 
delay of the communications system. The image presented to the operator remains 
substantially the same size and on substantially the same portion of the display, 
making efficient use of the display surface and providing a higher-resolution image to 
the operator than the Third Embodiment of lida et aL c 878. The operator's perception 
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is thus substantially equivalent to seeing what is seen by the camera, which is a 
significant aid in controlling the positioning of the camera and any vehicle upon 
which the camera may be mounted. 

While continuing to traverse the Examiner's rejections, Applicant has, in order 
to expedite the prosecution, chosen to amend independent claims 1 and 1 1 in order to 
clarify and emphasize the crucial distinctions between the present invention and the 
application of lida et al. '878 cited by the Examiner. Specifically, claims 1 and 11 
have been amended to clarify that the second image, which is a result of an estimate 
of the image currently seen by the camera, based upon a previous image from the 
camera and the movement commands that have been issued until this time, and, 
optionally, filler image data and/or historic image data, when available, is displayed 
upon substantially the same region of the display upon which the first image had been 
displayed. This is in contradistinction to the Third Embodiment of lida et al. c 878, in 
which the displayed image is moved from place to place on the display, corresponding 
to the direction the camera is expected to be pointing after the movement commands 
have been executed. 

Support for these amendments can be found in the specification. Specifically, 
support for the second image occupying substantially the same portion of the display 
as the first image can be found in Figures 1 and 2 and the following citation from 
page 7 lines 3-7: 

The current predicted view from camera 18 after the movement 
command is complete, is formed by manipulating image 28, for 
example by moving or scaling or rotating image 28? as appropriate, 
such that, the center of an image of a view from camera 18 at the 
second position corresponds to the center of predictive display 12. 
(emphasis added) 
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Amended independent claims 1 and 1 1 now feature language which makes it 
absolutely clear that the present invention provides for a predictive display having 
features not hinted at or suggested in Iida et al. 4 878. Applicant believes that the 
amendment of the claims completely overcomes the Examiner's rejections on § 102(e) 
grounds. 

To further distinguish the present invention Applicant has added claims 13 and 

14, which address the use of filler image data, and, more specifically, filler image data 

that are manipulated in a manner substantially equivalent to the manipulations that are 

applied to the first image data. This serves to provide the operator with an improved 

impression of continuity of the displayed images. 

Support for these new claims can be found in the specification. Specifically, 

support for manipulating the filler image data in a manner substantially equivalent to 

the manipulations applied to the first image data can be found in the following citation 

from page 8 lines 17-23: 

Filler section 36 is typically a static pattern which is 
manipulated in the same way as image 34, for example, by translation, 
rotation or scaling. Filler section 36 is generally a repetitive pattern 
such as a grid. Optionally, additional historic image data of the 
predicted view is used to fill in the filler section of image 34 instead of 
using a blank filler or a repetitive pattern filler. The historic image data 
is based on previous images at or close to the second position, 
(emphasis added) 

Amendments to the Specification 

No new matter has been added. 
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In view of the above amendments and remarks it is respectfully submitted that 
independent claims 1 and 11, and hence dependent claims 2-10 and 12-14 are in 
condition for allowance. Prompt notice of allowance is respectfully and earnestly 
solicited, 

Respectfulh/submitted, 

Mark/Kl Friedman 
AtJ^rney for Applicant 
Registration No. 33,883 

Date: July 18, 2007 
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